User-Centred Requirements Handbook |
A survey involves administering a set of written questions to a large sample population of users. Surveys can help determine information or customers, work practice and attitudes. There are two types: 'closed', where the respondent is asked to select from available responses and 'open', where the respondent is free to answer as they wish.
Typical Application Areas
Useful for obtaining quantitative data from users about existing tasks or the current system.
Benefits
Quick and relatively inexpensive to administer but not to design. Results can be subjected to statistical analysis.
Provides 'hard' data to supplement more subjective, qualitative information such as unstructured opinions.
Limitations
Survey design is not straightforward; many think they can do it but few laymen do it well - experienced guidance is needed. It may be hard to follow up on interesting comments as it is often not desirable or possible to keep records of respondents.
What you need
Depends very much on the complexity of the survey and the number of respondents needed.
Process
1. Initial steps are the same as for interview design, keeping in mind that semi-structured interviews are similar to open-ended surveys (i.e., the issues are known, but the range of user responses to them is not); and structured interviews are similar to closed-ended surveys (i.e., the range of user responses is pretty well understood, but the strength of each response category needs to be determined).
2. User sampling should be used and, if done properly, surveys should employ a rigorous statistical sampling method to ensure that results are not biased. However, this recommendation is rarely if ever observed in industry. It is sometimes done to offer respondents a little gift in exchange for a returned survey: if chosen appropriately, this can raise response rates to 80% and above. A low response rate may be followed up with either a re-posting or better still a telephonic contact. However, these methods require that users be identified by name to the researcher at least: some surveys may require total anonymity. It is usual to include a short covering letter requesting the respondent to reply and a stamped addressed envelope if possible to make the return as easy for the respondent as can be.
3. If user information is being kept on computer (as is almost inevitable these days) care should be taken to ensure that the data privacy legislation in your country is not breached, and respondents should be assured of this in the covering letter.
Practical guidelines
• Explain the aim of the survey at the beginning of the form. If it is a postal survey include a polite covering letter explaining its purpose. If the respondents can see the reason for completing the survey, they are more likely to do so.
• If the survey is to be returned by post, make it easy to return by providing a pre-addressed envelope (preferably stamped).
• Avoid too many open-ended questions as this will increase the analysis effort required.
• Make sure that multiple choice answers are mutually exclusive.
• Make sure that the question wording is clear and concise.
• Avoid double questions in a single question.
• Avoid questions involving negatives.
• Avoid complex branching structures.
• Avoid asking questions that make users feel uncomfortable or offend. For example it may be better to ask for selection of an age band rather than exact age.
• For visually impaired users provide the survey either in large print form, Braille, or allow the respondent to provide their responses via a face to face interview or over the phone.
• Provide mentally impaired users the option to respond to the survey via an interview.
• Motor impaired users may have trouble filling in the survey. The provision of large completion boxes will assist them. Again they may prefer to be interviewed.
• Young users may be able to answer the questions but not be able to read. They may therefore also require the survey to be conducted as an interview.
• For young users able to read, make sure that the questions are simple and understandable.
Further information
Preece (1994).
Refer to RESPECT deliverable D6.2 for information on performing surveys
with users who have impairments and disabilities, as well as elderly
and young users.